Method of processing information, and information processing apparatus

ABSTRACT

A method of processing information includes: identifying a time span in a period of viewing a content based on detection results of the behavioral viewing states of the user viewing the content, the time span being a period, during which a behavioral viewing state of the user is not determined to be a positive state or a negative state; extracting a time period during which an index indicating one of the positive state and the negative state of the user has an unordinary value with respect to values of the other time periods in the time span; and estimating a time period, during which the user has quite possible been in at least one of the positive state and the negative state, based on the time period extracted by the extracting.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority of theprior Japanese Patent Application No. 2013-118337, filed on Jun. 4,2013, the entire contents of which are incorporated herein by reference.

FIELD

The embodiments discussed herein are related to a method of processinginformation, and an information processing apparatus.

BACKGROUND

To date, proposals have been made on techniques for determining a degreeof attention of a user to a content using a biosensor and a capturedimage. In the case of using a biosensor, methods of determining a degreeof concentration of a user based on GSR (Galvanic Skin Response), Skintemperature, and BVP (Blood Volume Pulse) have been known. In the caseof using an image, methods of calculating a degree of enthusiasm of auser by a posture of the user (leaning forward and leaning back) havebeen known.

As examples of related-art techniques, Japanese Laid-open PatentPublication Nos. 2003-111106 and 2006-41887 have been known.

SUMMARY

According to an aspect of the invention, a method of processinginformation includes: identifying a time span in a period of viewing acontent based on detection results of the behavioral viewing states ofthe user viewing the content, the time span being a period, during whicha behavioral viewing state of the user is not determined to be apositive state or a negative state; extracting a time period duringwhich an index indicating one of the positive state and the negativestate of the user has an unordinary value with respect to values of theother time periods in the time span; and estimating a time period,during which the user has quite possible been in at least one of thepositive state and the negative state, based on the time periodextracted by the extracting.

The object and advantages of the invention will be realized and attainedby means of the elements and combinations particularly pointed out inthe claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 schematically illustrates a configuration of an informationprocessing system according to a first embodiment;

FIG. 2A illustrates a hardware configuration of a server;

FIG. 2B illustrates a hardware configuration of a client;

FIG. 3 is a functional block diagram of the information processingsystem;

FIG. 4 illustrates an example of a data structure of an attentiveaudience sensing data DB;

FIG. 5 illustrates an example of a data structure of a displayedcontents log DB;

FIG. 6 illustrates an example of a data structure of a user'saudio/visual action state variable DB;

FIG. 7 illustrates an example of a data structure of a user statusdetermination result DB;

FIG. 8 illustrates an example of a data structure of an unordinary PNNtransition section DB;

FIG. 9 is a flowchart of attentive audience sensing data collection andmanagement processing executed by a data collection unit;

FIG. 10 is a flowchart of posture change evaluation value calculationprocessing executed by the data collection unit;

FIG. 11 is a flowchart of total viewing area calculation processingexecuted by the data collection unit;

FIGS. 12A, 12B, 12C, 12D and 12E are explanatory diagrams of theprocessing illustrated in FIG. 11;

FIG. 13 is a flowchart illustrating processing of a data processing unitaccording to the first embodiment;

FIG. 14 is a graph illustrating a PNN transition value in anaudio/visual digital content timeframe in which a user state isdetermined to be neutral;

FIGS. 15A and 15B are explanatory diagrams of advantages of the firstembodiment;

FIGS. 16A, 16B and 16C are explanatory diagrams of an example ofgrouping according to a second embodiment;

FIG. 17 is a flowchart illustrating processing of a data processing unitaccording to the second embodiment;

FIG. 18 is a table illustrating determination results of unordinary PNNtransition section for each individual groups, which are totaled inaudio/visual digital content timeframes t_(m1) to t_(m1+n);

FIG. 19 is a flowchart illustrating processing of a data processing unitaccording to a third embodiment;

FIG. 20 is a flowchart illustrating processing of a data processing unitaccording to a fourth embodiment;

FIG. 21 is a table that is generated by the processing illustrated inFIG. 20;

FIG. 22 is a diagram illustrating an example of a PNN transition valueaccording to a fifth embodiment; and

FIG. 23 is a table illustrating an example of comparison of PNNtransition values and determination results of an unordinary PNNtransition section in multiple constant period according to the fifthembodiment.

DESCRIPTION OF EMBODIMENTS

In the case of using a captured image in order to determine a degree ofattention of a user to a content, when it is expected that the user willbe get excited very much as a characteristic of the content to bedisplayed, it is possible to determine the state of the user using asimple model. For example, if the user has an interest or attention, theuser tends to lean forward, whereas if the user has no interest orattention, the user tends to lean back.

However, when a content to be viewed is not a content that is notexpected to gets the user excited very much, such as e-learning or alecture video, it is difficult to determine whether the user has aninterest or attention from his or her appearance/behavior. Accordingly,it is also difficult to definitely determine whether the user isinterested or not from the captured images of the state of a user.

According to an embodiment of the present disclosure, it is desirable toprovide a method of processing information, and an informationprocessing apparatus that allow a precise estimation of the state of auser who is viewing a content.

First Embodiment

In the following, a detailed description will be given of a firstembodiment of an information processing system with reference to FIG. 1to FIG. 15. An information processing system 100 according to the firstembodiment is a system in which estimation is made of a state of a userwho has viewed a content in order to utilize the estimation result. Forexample, an estimation is made of a time period in which the user was ina state (positive) of having an interest or attention in the content, atime period in which the user in a state (negative) of not having aninterest or attention in the content, or a time period in which the userhad a high possibility of having been interested or paying attention tothe content, and so on.

FIG. 1 illustrates a schematic configuration of the informationprocessing system 100. As illustrated in FIG. 1, the informationprocessing system 100 includes a server 10 as an information processingapparatus, clients 20, and information collection apparatuses 30. Theserver 10, the clients 20, and the information collection apparatuses 30are connected to a network 80, such as the Internet, a local areanetwork (LAN), and so on.

The server 10 is an information processing apparatus on which serversoftware is deployed (installed), and which aggregates, provides indexand stores, and classifies data on the user states, and performs variouscalculations on the data. Also, the server 10 manages contentdistribution including control of the display mode of the content to bedistributed to the users. FIG. 2A illustrates a hardware configurationof the server 10. As illustrated in FIG. 2A, the server 10 includes acentral processing unit (CPU) 90, a read only memory (ROM) 92, a randomaccess memory (RAM) 94, a storage unit (here, a hard disk drive (HDD))96, a display 93, an input unit 95, a network interface 97, a portablestorage medium drive 99, and so on. Each of these components of theserver 10 is connected to a bus 98. The display 93 includes a liquidcrystal display, or the like, and the input unit 95 includes a keyboard,a mouse, a touch panel, and so on. In the server 10, the CPU 90 executesprograms (including an information processing program) that are storedin the ROM 92 or the HDD 96, or programs (including an informationprocessing program) that are read by the portable storage medium drive99 from the portable storage medium 91 so as to achieve functions as acontent management unit 50, a data collection unit 52, and a dataprocessing unit 54, which are illustrated in FIG. 3. In this regard,FIG. 3 illustrates a content database (DB) 40, an attentive audiencesensing data database (DB) 41, a displayed contents log database (DB)42, a user's audio/visual action state variable database (DB) 43, a userstatus determination result database (DB) 44, and an unordinary PNNtransition section database (DB) 45, which are stored in the HDD 96 ofthe server 10, and so on. In this regard, descriptions will be givenlater of specific data structures, and so on of the individual DBs 40,41, 42, 43, 44, and 45.

The content management unit 50 provides a content (for example, alecture video) stored in the content DB 40 to the client 20 (a displayprocessing unit 60) in response to a demand from a user of the client20. The content DB 40 stores data of various contents.

The data collection unit 52 collects information on a content that theuser has viewed from the content management unit 50, and collects dataon a state of the user while the user is viewing the content from theinformation collection apparatus 30. Also, the data collection unit 52stores the collected information and data into the attentive audiencesensing data DB 41, the displayed contents log DB 42, and the user'saudio/visual action state variable DB 43. In this regard, the datacollection unit 52 collects attentive audience sensing data, such as auser monitoring camera data sequence, an eye gaze tracking (visualattention area) data sequence, a user-screen distance data sequence and,so on as data on the state of the user.

The data processing unit 54 determines a timeframe in which the user whohas been viewing the content was in a positive state and a timeframe inwhich the user was in a negative state based on the data stored in thedisplayed contents log DB 42 and the data stored in the user'saudio/visual action state variable DB 43. Also, the data processing unit54 estimates a timeframe (unordinary PNN transition section) having ahigh possibility of the user having been in a positive state. The dataprocessing unit 54 stores a determination result and an estimationresult in the user status determination result DB 44 and the unordinaryPNN transition section DB 45, respectively.

The client 20 is an information processing apparatus in which clientsoftware is disposed (installed), and is an apparatus which displays andmanages a content for the user, and stores viewing history. As theclient 20, it is possible to employ a mobile apparatus, such as a mobilephone, a smart phone, and the like in addition to a personal computer(PC). FIG. 2B illustrates a hardware configuration of the client 20. Asillustrated in FIG. 2B, the client 20 includes a CPU 190, a ROM 192, aRAM 194, a storage unit (HDD) 196, a display 193, an input unit 195, anetwork interface 197, a portable storage medium drive 199, and thelike. Each component in the client 20 is connected to a bus 198. In theclient 20, the CPU 190 executes the programs stored in the ROM 192 orthe HDD 196, or the programs that are read by the portable storagemedium drive 199 from the portable storage medium 191 so that thefunctions as the display processing unit 60 and the input processingunit 62, which are illustrated in FIG. 3, are achieved.

The information collection apparatus 30 is an apparatus for collectingdata on a state of the user, and includes a Web camera, for example. Theinformation collection apparatus 30 transmits data on the collected userstates to the server 10 through the network 80. In this regard, theinformation collection apparatus 30 may be incorporated in a part of theclient 20 (for example, in the vicinity of the display).

In this regard, in the present embodiment, in the client 20, informationdisplay in accordance with a user is performed on the display 193. Also,information collection from a user of the client 20 is stored in theinformation processing system 100 in association with the user. That isto say, the client 20 is subjected to access control by client softwareor server software. Accordingly, in the present embodiment, informationcollection and information display are performed on the assumption ofthe confirmation that the user of the client 20 is “user A”, forexample. Also, in the client 20, information display is performed inaccordance with the display 193. Also, information collected from theclient 20 is stored in the information processing system 100 inassociation with the display 193. That is to say, information collectionand information display are performed on the assumption that the client20 is recognized by the client software or the server software, and thedisplay 193 is confirmed to be a predetermined display (for example,screen B (scrB)).

Next, descriptions will be given of the data structures of the DBs 41,42, 43, 44, and 45 accessed by the server 10 with reference to FIGS. 4,5, 6, 7, and 8.

FIG. 4 illustrates an example of a data structure of the attentiveaudience sensing data DB 41. The attentive audience sensing data DB 41stores attentive audience sensing data that has been collected by thedata collection unit 52 through the information collection apparatus 30.As illustrated in FIG. 4, the attentive audience sensing data DB 41stores “user monitoring camera data sequence”, “eye gaze tracking(visual attention area) data sequence”, and “user-screen distance datasequence” as an example.

The user monitoring camera data sequence is a recorded state of the userwhile the user was viewing a content as data. Specifically, the usermonitoring camera data sequence includes individual fields such as“recording start time”, “recording end time”, “user”, “screen”, and“recorded image”, and stores the state of the user (A) who is viewingthe content displayed on the display (for example, screen B) in a movingimage format. In this regard, in the user monitoring camera datasequence, an image sequence may be stored.

The eye gaze tracking (visual attention area) data sequence stores a“visual attention area estimation map”, which is obtained by observingan eye gaze of the user at each time, for example. In this regard,although it is possible to achieve observation of the eye gaze using aspecial device that measures an eye movement. However, it is possible todetect user's gaze location using a marketed Web camera (assumed to bedisposed at the upper part of the display (screen B) of the client 20)included in the information collection apparatus 30 (for example, referto Stylianos Asteriadis et al., “Estimation of behavioral user statebased on eye gaze and head pose-application in an e-learningenvironment”, Multimed Tools Appl, 2009). In these gaze detectiontechniques, visually focused data focused on a content is represented asdata called a heat map in which a time period during which visualfocused points is represented by an area of a circle and intensity ofoverlay (refer to FIG. 12D). In the present embodiment, a circle thatcontains all the plurality of positions on which visual focused pointsis recorded as visually focused point information for each of the unittime with a center of gravity of a plurality of positions on whichvisual focused points per unit time as a center. Here, in a visualattention area estimation map recorded at time t₁, circle informationthat has been recorded since time t₀ is recorded in the file thereof. Inthe case where five pieces of visually focused point information isrecorded from time t₀ to time t₁, up to five pieces of circleinformation is recorded (refer to FIG. 12D).

The user-screen distance data sequence records, for example, a distancemeasurement result between a user (A) who is viewing a content and ascreen (screen B) on which the content is displayed. The distancemeasurement may be performed using a distance sensing special device,such as a laser beam or a depth sensing camera. However, a usermonitoring camera data sequence captured by a Web camera included in theinformation collection apparatus 30 may be used as a measurement. If auser monitoring camera data sequence captured by a Web camera is used,it is possible to use an accumulated value of a change of the size of arectangular area enclosing a face of the user multiplied by acoefficient as a distance.

FIG. 5 illustrates an example of a data structure of the displayedcontents log DB 42. The user sometimes views a plurality of screens atthe same time, or sometimes displays a plurality of windows on onescreen. Accordingly, the displayed contents log DB 42 stores in whatstate windows are disposed on a screen, and which content is displayedin a window as a history. Specifically, as illustrated in FIG. 5, thedisplayed contents log DB 42 includes tables of “screen coordinates”,“displayed content log”, and “content”.

The “screen coordinates” table records “overlaying order” of windows inaddition to “window IDs” of the windows displayed on a screen,“upper-left x coordinate” and “upper-left y coordinate” indicating adisplay position, “width” and “height” indicating a size of a window. Inthis regard, the reason why “overlaying order” is recorded is that evenif a content is displayed in a window, there are cases where a window ishidden under another window, and thus a window that is not viewed by theuser sometimes exists on a screen.

The “displayed content log” table records “window IDs” of the windowsthat are displayed on a screen, and “content ID” and “content timeframe”of a content that is displayed on the windows. The “content timeframe”records which section (timeframe) of a content is displayed. In thisregard, for “content ID”, the content ID defined in “content” table inFIG. 5 is stored.

FIG. 6 illustrates an example of a data structure of the user'saudio/visual action state variable DB 43. The user's audio/visual statevariable is used as a variable of the user state. In FIG. 6, the user'saudio/visual action state variable DB 43 has tables of “movement of faceparts”, “eye gaze”, and “posture change” as an example.

The “movement of face parts” table includes fields, such as “time(start)”, “time (end)”, “user”, “screen”, “blinking”, “eyebrow”, . . . ,and so on. The “movement of face parts” table stores information, suchas whether blinking and eyebrow movement was active, moderate, and so onduring the time from “time (start)” to “time (end)”.

The “eye gaze” table includes fields of “time (start)”, “time (end)”,“user”, “screen”, “window ID”, “fixation time (msec)”, and “total areaof viewed content”. Here, the “fixation time (msec)” means a time periodduring which a user fixates his/her eye gaze. Also, “total area ofviewed content” means a numeric value representing the amount of eachcontent viewed by the user, and the details thereof will be describedlater.

The “posture change” table includes fields of “time (start)”, “time(end)”, “user”, “screen”, “window ID”, “user-screen distance”, “posturechange flag”, and “posture change time”. The detailed descriptions willbe given later of “user-screen distance”, “posture change flag”, and“posture change time”, respectively.

FIG. 7 illustrates an example of a data structure of the user statusdetermination result DB 44. The user status determination result DB 44records in which status the user was for each content timeframe. For auser status, if in a positive state, “P” is recorded, if in a negativestate, “N” is recorded, and if in a state which is neither positive nornegative (neutral state), “-” is recorded.

FIG. 8 illustrates an example of a data structure of the unordinary PNNtransition section DB 45. The unordinary PNN transition section DB 45records a timeframe (unordinary PNN transition section) that is allowedto be estimated as highly possible to be positive although in a statewhich is neither positive nor negative (neutral state) in the userstatus determination result DB 44. A description will be later given ofspecific contents of the unordinary PNN transition section DB 45.

Next, a description will be given of processing that is executed on theserver 10.

Attentive Audience Sensing Data Collection/Management Processing

First, a description will be given of attentive audience sensing datacollection/management processing executed by the data collection unit 52with reference to a flowchart in FIG. 9.

In the processing in FIG. 9, first, in step S10, the data collectionunit 52 obtains attentive audience sensing data from the informationcollection apparatus 30. Next, in step S12, the data collection unit 52stores the obtained attentive audience sensing data into the attentiveaudience sensing data DB 41 for each kind (for each user monitoringcamera data sequence, eye gaze tracking (visual attention area) datasequence, and user-screen distance data sequence).

Next, in step S14, the data collection unit 52 calculates a user'saudio/visual action state variable at viewing time in accordance with atimeframe of the audio/visual digital contents, and stores the user'saudio/visual action state variable into the user's audio/visual actionstate variable DB 43.

Here, various kinds of processing is assumed in accordance with thekinds of data collected as step S14. In the present embodiment,descriptions will be given of processing for calculating the posturechange flag and the posture change time in the “posture change” table inFIG. 6 (the posture change evaluation value calculation processing), andprocessing for calculating the total area of viewed content in the “eyegaze” table in FIG. 6 (total viewing area calculation processing).

Posture Change Evaluation Value Calculation Processing

A description will be given of posture change evaluation valuecalculation processing executed by the data collection unit 52 withreference to a flowchart with reference to FIG. 10.

In the processing in FIG. 10, first, in step S30, the data collectionunit 52 obtains a content timeframe i. In this regard, it is assumedthat the length of the content timeframe i is Ti.

Next, in step S32, the data collection unit 52 ties a content timeframeand attentive audience sensing data from the displayed contents log DB42, the user-screen distance data sequence in the attentive audiencesensing data DB 41, and creates a posture change table as a user'saudio/visual action state variable. In this case, data is input intoeach field of “time (start)”, “time (end)”, “user”, “screen”, “windowID”, and “user-screen distance” of the posture change table in FIG. 6.In this regard, data of the amount of change from a reference position(a position where the user is ordinary positioned) is input in the fieldof “user-screen distance”.

Next, in step S34, the data collection unit 52 determines whether auser-screen distance |d| is greater than a threshold value. Here, it ispossible to employ a reference position×(1/3)=50 mm, and so on for thethreshold value, for example.

In step S34, if it is determined that the user-screen distance is largerthan the threshold value, the processing proceeds to step S36, and thedata collection unit 52 sets the posture change flag to “true” in theposture change table. Also, the data collection unit 52 sets posturechange time T_(trans) _(—) _([i]) of the content timeframe i as follows:T_(trans) _(—) _([i])=T_(trans) _(—) _([i])+Ti. In this regard, in theposture change table in FIG. 6, the posture change flag of the seconddata, and the third data from top becomes “true”, and 600 (msec) in thetimeframe 2012/7/12 11:00:01.69 to 2012/7/12 11:00:02.29 is recorded asposture change time.

On the other hand, in step S34, if it is determined that the user-screendistance is less than the threshold value, the processing proceeds tostep S38, and the data collection unit 52 sets the posture change flagto “false” in the posture change table. Also, the data collection unit52 sets the posture change time T_(trans) _(—) _([i]) as follows:T_(trans) _(—) _([i])=0 (for example, refer to the fourth data from thetop in the posture change table in FIG. 6).

Next, in step S40, the data collection unit 52 determines whether thereremains a content timeframe whose posture change evaluation value is tobe calculated. Here, if it is determined that there remains a contenttime frame whose posture change evaluation value is to be calculated,the processing returns to step S30, and the above-described processingis repeated, otherwise all the processing in FIG. 10 is terminated.

Total Viewed Area Calculation Processing

Next, a description will be given of processing for calculating “totalarea of viewed content” in the eye gaze table in FIG. 6, which isexecuted by the data collection unit 52, (total viewed area calculationprocessing) with reference to a flowchart in FIG. 11. Here, adescription will be given of the case of calculating the total viewedarea of the content having a content ID=Z (=1) displayed in the windowhaving the window ID=Y (=1) on the screen X (=scrB) at time t1.

In this regard, it is assumed that, as illustrated in FIG. 12A, thereare two windows on the screen B at time t₀, and as illustrated in FIG.12B, there is one window on the screen B at time t1. Here, there is norecord as to at what timing from time t₀ to t₁, the window having thewindow ID: 2 has disappeared. Accordingly, it is not possible tocorrectly calculate the amount of time during which the contents havingthe content ID=1 and content ID=2, which are displayed in the window ID:1 and the window ID: 2, respectively, were viewed. Accordingly, in thepresent embodiment, on the assumption that a time period from time t₀ tot₁ is very small, the total viewed area at the point in time t₁, iscalculated only on a content that is displayed in the window existing attime t₁.

In the processing in FIG. 11, first, in step S50, the data collectionunit 52 calculates an area S_(DW) of a window region t₀ to t₁. Here,“window region t₀ to t₁” means a region that is bounded by outerperipheral edges of all the windows displayed on the screen B duringtime from time t₀ to time t₁. Specifically, it is assumed that the stateof the screen B at time t₀ is a state as illustrated in FIG. 12A, andthe state of the screen B at time t₁ is a state as illustrated in FIG.12B. In this case, the window region t₀ to t₁ is a region that isbounded by outer peripheral edges of the window ID: 1 displayed, andouter peripheral edges of the window ID: 2 displayed, which means aregion illustrated by a bold line in FIG. 12C.

Next, in step S52, the data collection unit 52 calculates an area S_(1W)of the window of the window ID: 1 at time t₁. Here, a window area of thewindow ID: 1 illustrated in FIG. 12B is calculated.

Next, in step S54, the data collection unit 52 calculates the total area(in addition to the overlapped area) S_(ES) of the visual attention areaestimation map included in the window region t₀ to t₁. Here, if it isassumed that five circles are recorded in the visual attention areaestimation map at time t₁ as in FIG. 12D, as the area S_(ES), an area ofan overlapping portion with the window region t₀ to t₁ as illustrated inFIG. 12E is calculated among the five circles.

Next, in step S56, the data collection unit 52 calculates the totalviewed area S of the content of the content ID=1 displayed in the windowof the window ID: 1 on the screen B at the current time t₁. In thiscase, it is possible to calculate the total viewed area S allocated tothe content of the content ID=1, which is displayed in the window of thewindow ID: 1 as the product of the area S_(ES) and the ratio of thewindow area at time t₁ to the region bounded by outer peripheral edgesof all the windows displayed during the time from time t₀ to time t₁,S_(1W)/S_(DW) (allocation ratio). That is to say, it is possible for thedata collection unit 52 to calculate the area S by Expression (1).

S=S _(ES) ×S _(1W) /S _(DW)  (1)

In this regard, FIG. 6 illustrates an example in which “8000” iscalculated as a total area of viewed content.

Processing of Data Processing Unit 54

Next, a description will be given of processing of the data processingunit 54 with reference to a flowchart in FIG. 13. In this regard, thedata processing unit 54 may perform the processing in FIG. 13 in sametime phase with the calculation (FIG. 9) of the user's audio/visualaction state variable, or may start the processing in same time phasewith the content viewing end. Alternatively, the data processing unit 54may perform the processing in FIG. 13 as periodical batch processing.

In the processing in FIG. 13, first, in step S70, the data processingunit 54 identifies a positive (P), a neutral (-), and a negative (N)content timeframes, and stores the timeframes into the user statusdetermination result DB 44. For example, it is assumed that when theuser status is positive at content viewing time, the posture change timebecomes short, and when the user status is negative, the posture changetime becomes long. In this case, it is possible to use the posturechange time as a user's audio/visual action state variable value. If itis assumed that the user's audio/visual action state variable valueS=posture change time T_(trans), it is possible to represent a userstatus U in the form of U={Positive (S<2000), Negative (S>4000)}.Alternatively, it is possible to assume that if the user status ispositive at content viewing time, the fixation time of the eye gaze islong, whereas if the user status is negative, the fixation time of theeye gaze is short. In this case, it is possible to use the fixation timeof the eye gaze as the user's audio/visual action state variable value.If it is assumed that the user's audio/visual action state variablevalue S=fixation time of eye gaze T_(s), it is possible to represent asin the form that the user status U={Positive (S>4000), Negative(S<1000)}. In this regard, in the first embodiment, a description willbe given of the case of using the posture change time as the user'saudio/visual action state variable value.

Next, in step S72, the data processing unit 54 extracts a timeframe(called a PNN section) in which a change occurs: positive→negative.Here, in the PNN section (during the content timeframes 5 to m in FIG.7), although it is not possible to determine the user status to bepositive nor negative, it is estimated that the value of the user'saudio/visual action state variable value S monotonously increases withtime passage.

Next, in step S74, the data processing unit 54 identifies an unordinaryPNN transition section, and records it into the unordinary PNNtransition section DB 45. In this regard, the unordinary PNN transitionsection means, in this case, a timeframe in which although a monotonousincrease is estimated, a timeframe that indicates different transition.Also, in this timeframe, there is a possibility that an unordinary stateof the user has occurred contrary to ordinary transition from positiveto negative. Accordingly, in the present embodiment, this unordinary PNNtransition section is regarded as a timeframe that has a highpossibility of a timeframe (neutral plus section) in which the userbecomes the positive state compared with the neighboring timeframe.

Specifically, in step S74, the data processing unit 54 records a contenttimeframe i in which although S_(i)>S¹⁻¹ (S_(i): PNN transition value ofthe content timeframe i) is ordinarily supposed to hold in the PNNsection, S_(i)<S¹⁻¹ holds as an unordinary PNN transition section. Inthis regard, the PNN transition value means an index value indicating apositive state or a negative state of the user. In FIG. 14, the PNNtransition values S₆ to S_(m−1) between the content timeframes (PNNsections) 6 and (m−1) in which the user status is determined to beneutral, illustrated in FIG. 8, is represented by a line graph (solidline). Also, in FIG. 14, a line that linearly approximates a monotonousincrease change of the PNN section is represented by a dashed-dottedline.

Here, in step S74, a degree of being out of ordinary in a certaintimeframe is represented by a numeric value. As a numeric value in thiscase, it is possible to employ a degree of being out of ordinarytimeframe represented by the difference between the linearlyapproximated value and the calculated value. In this case, if it isassumed that the absolute value differences between each PNN transitionvalue and a linearly approximated value in the adjacent timeframes ofthe content timeframe i−1, the content timeframe i, and the contenttimeframe i+1 are e_(i−1), e_(i), and e_(i+1), respectively, it ispossible to represent the degree of being out of ordinary in the contenttimeframe i by the difference with the absolute value differenceordinarily assumed. Here, if it is assumed that the assumed absolutevalue difference is (e_(i−1)+e_(i+1))/2, the degree of being out ofordinary timeframe (degree of unordinary timeframe) q_(i) of the contenttimeframe i is represented by e_(i)−((e_(i−1)+e_(i+1))/2). Accordingly,it is possible for the data processing unit 54 to obtain the degree ofbeing out of ordinary timeframe q_(i) in each neutral plus sectioncandidate (each timeframe in the PNN section) as a neutral plusestimation value v_(i), and to determine that the timeframe is neutralplus section if a value v_(i) is a certain value or more. In thisregard, in the present embodiment, a timeframe indicated by an arrow inFIG. 14 (the content timeframe=9 in FIG. 8) is recorded as an unordinaryPNN transition section (neutral plus section).

In this manner, in the present embodiment, as illustrated in FIG. 15A,even if there is an intermediate timeframe a between positive andnegative (a timeframe that is neither positive nor negative), asillustrated in FIG. 15B, it is possible to represent a timeframe havinga high possibility of the user becoming positive state by a neutral plusestimation value v_(i). Also, in the present embodiment, it is possibleto estimate a neutral plus section (unordinary PNN transition section)based on the neutral plus estimation value v_(i), and thus it ispossible to estimate the user state with high precision.

In this regard, the server 10 uses the data stored or recorded in theuser status determination result DB 44 and the unordinary PNN transitionsection DB 45 as described above. Accordingly, for example, it ispossible to create an abridged version of a content using content imagesof a positive section and a neutral plus section, and so on. Also, it ispossible to provide (feedback) a content creator (a lecturer, and so on)with information on timeframes in which the user had an interest orattention, and timeframes in which the user had no interest orattention, and so on. In this manner, in the present embodiment, it ispossible to evaluate, and reorganize a content, and to performstatistics processing on a content, and so on.

In this regard, in the present embodiment, the data processing unit 54achieves functions of an identification unit that identifies a PNNsection from a detection result of a positive state and a negative stateof a user who is viewing a content, an extraction unit that extracts atimeframe in which an index value (PNN transition value) representing apositive or negative state of the user indicates an unordinary value inthe identified PNN section, and an estimation unit that estimates theextracted timeframe as a neutral plus section.

As described above in detail, by the first embodiment, the dataprocessing unit 54 identifies a timeframe (PNN section) in which theuser status is allowed to be identified as neither positive (P) nornegative (N) in the detection result of the state of the user viewing acontent, extracts a time period in which the PNN transition value of theuser indicates an unordinary value (a time period indicating differenttendency from a monotonous increase or a monotonous decrease) in theidentified PNN section, and estimates that the extracted time period isa time period (neutral plus section) having a high possibility of theuser having been in a positive or a negative state. Thereby, even in thecase where it is difficult to determine whether the user has an interestor attention (a case of viewing e-learning or a moving image of alecture, and so on), it is possible to estimate a timeframe having ahigh possibility that the user had an interest or attention.

Also, in the present embodiment, it is possible to estimate a timeframehaving a high possibility of the user having an interest or attentionusing a simple apparatus, such as a Web camera, and so on. Accordingly,a special sensing device does not have to be introduced, and thus it ispossible to reduce cost.

In this regard, in the above-described embodiment, a description hasbeen given of the case where a positive or negative section isidentified using the attentive audience sensing data in step S70 in FIG.13. However, the present disclosure is not limited to this, and the dataprocessing unit 54 may identify a positive or negative section usingdata obtained by directly hearing from the user.

Also, in the above-described embodiment, a description has been given ofthe case of using one kind of data as a user's audio/visual action statevariable. However, the present disclosure is not limited to this, anduser's audio/visual action state variables of a plurality of persons maybe used in combination (for example, representing by a polynomial, andso on).

In this regard, in the above-described embodiment, a description hasbeen given of the case where an unordinary section is estimated in atimeframe of changing from positive to negative (PNN section). However,the present disclosure is not limited to this, and the unordinarysection may be estimated in a timeframe of changing from negative topositive.

Second Embodiment

In the following, a description will be given of a second embodimentwith reference to FIG. 16 and FIG. 17. In the second embodiment, thedata processing unit 54 in FIG. 3 divides a plurality of users into aplurality of groups, and performs determination processing of theunordinary PNN transition section of a content based on the state (apositive state or a negative state) of the users who belong to thatgroup.

In the case where a plurality of users view a content (for example,e-learning and a lecture video), it is possible for the data collectionunit 52 to collect a large amount of data of the user states at contentviewing time. Accordingly, it is possible for the data processing unit54 to classify users who have completed viewing into several groupsusing the large amount of collected user's audio/visual action statevariables. FIGS. 16A, 16B and 16C illustrates an example of classifyinguser groups in the case of using posture changes as the user'saudio/visual action state variable. One vertical bar in FIG. 16illustrates a posture change time in a certain timeframe. In thisclassification example, attention is focused on transition tendency froma timeframe (positive) having a short posture change time to a timeframe(negative) having a long posture change time: positive neutral negative.FIG. 16A illustrates an example of a group having a long neutral time,FIG. 16B illustrates an example of a group having a long negative time,and FIG. 16C illustrates an example of a group having a long positivetime.

In this regard, the data processing unit 54 may perform grouping usingdata of all the timeframes during viewing of one content at the time ofgrouping, or may perform grouping on the users having transition of userstates by focusing attention on a specific timeframe of a specificcontent in the same manner. Alternatively, the data processing unit 54may perform grouping by a common posture change pattern (the number ofposture change times significantly decreases in the latter half of acontent, and so on) without identifying a content. In either case, usershaving similar tendency of user's audio/visual action state variabletransition are grouped together.

In the following, a description will be given of the processing of thedata processing unit 54 according to the second embodiment withreference to a flowchart in FIG. 17.

In this regard, in the second embodiment, it is assumed that n neutralplus estimation values are calculated from user A during a PNN section k(content ID=1, content timeframes t_(m1) to t_(m1+n)). Also, it isassumed that user A is classified into a user group 1 (neutral time islong).

First, in step S100, the data processing unit 54 obtains the number ofusers C_(max) of the group to which the user A belongs.

Next, in step S102, the data processing unit 54 obtains an unobtainedcontent timeframe i among content timeframes t_(m1) to t_(m1+n). Next,in step S104, the data processing unit 54 obtains the number of usersC_(i) whose content timeframe i is an unordinary PNN transition section.

Next, in step S106, the data processing unit 54 calculates the neutralplus estimation value v_(i) by Expression (2). In this regard, q_(i)means a degree of being out of ordinary timeframe of the user A in thecontent timeframe i.

v _(i) =q _(i)×(C _(i)+1)/C _(max)  (2)

As Expression (2), in the present embodiment, in the calculation of theneutral plus estimation value, a ratio of the number of users determinedto be in the unordinary PNN transition section in each content timeframein a group is used as a weight. Thereby, as a timeframe that isdetermined to have an unordinary PNN transition among the users in asame group in common, a higher value is calculated as a neutral plusestimation value. In this manner, in the present embodiment, in theover-all length of the PNN section, a timeframe in which there wasmovement determined to have positive tendency in common among the user Aand the group to which the user A belongs is estimated to be a neutralplus section so that it is possible to estimate a user state inconsideration of the group tendency.

After that, in step S108, a determination is made of whether all thecontent timeframes have been obtained. If it is determined that all thecontent timeframes have not been obtained, the processing returns tostep S102. And the processing and the determination after step S102 arerepeated. At the time when it is determined that all the contenttimeframes have been obtained in step S108, all the processing in FIG.17 is terminated.

In this regard, in the second embodiment, the data processing unit 54achieves the functions of the grouping execution unit that groups aplurality of users who view a content, the identification unit thatidentifies a PNN section in which a specific user (user A) state isdetermined to be neither positive nor negative, and the estimation unitthat determines, in the over-all length of the PNN section, a timeframein which there was movement determined to have positive tendency incommon among a specific user and the group to which the specific userbelongs, and estimates the timeframe to be a neutral plus section.

In the above, as described above, in the second embodiment, the dataprocessing unit 54 identifies a PNN section of the user A, andestimates, in the over-all length of the PNN section, a timeframe inwhich there was movement determined to have positive tendency in commonamong the user A and the group to which the user A belongs (a timeframehaving a high neutral plus estimation value v_(i)) to be a neutral plussection. Thereby, it is possible to estimate a user state inconsideration of group tendency having user state transition in the samemanner, and thus it is possible to make an estimation of minute userstate change with high precision.

Third Embodiment

Next, a description will be given of a third embodiment with referenceto FIG. 18 and FIG. 19. In the third embodiment, the data processingunit 54 performs correction of a neutral plus estimation value using notonly the information of the group to which the user belongs, but alsothe information of the other groups. In this regard, in the presentembodiment, it is assumed that the user A of the client 20 displays thecontent ID=1 on the screen B in the same manner as the secondembodiment. Also, in the present embodiment, a description will be givenof the case where n neutral plus estimation values of the PNN section k(the content timeframes t_(m1) to t_(m1+n)) are calculated.

In the present embodiment, the PNN transition values of the other groups2 and 3 are referenced using not only the group 1 to which the user A isclassified, but also data of the content timeframes t_(m1) to t_(m1+n).Here, FIG. 18 illustrates determination results of the unordinary PNNtransition section for each individual groups, which are totaled in thecontent timeframes t_(m1) to t_(m1+n). In FIG. 18, it is assumed thatthe degree of being out of ordinary of the content timeframe i of eachgroup is determined using the average value of the data of the users whobelong to each group. In this regard, as a method of obtaining degree ofbeing out of ordinary in the content timeframe i, a method of using alocal maximal value or a median, and the like of the data of the userswho belong to each group may be employed. Also, a numeric value, such as[0.1], and so on in FIG. 18 indicates a ratio of the users determined tobe an ordinary section when a determination of the unordinary PNNtransition section is performed using data of each single user in eachgroup. The determination of the unordinary section is performed usingthe user's audio/visual action state variables, and thus it is assumedthat there are cases where it is difficult for a group to be determinedto be the unordinary section. For example, as an example in posturechange, a case of a group having substantially no posture change in thePNN section, and so on.

In the third embodiment, the data processing unit 54 performs processingfor correcting biased determination among these groups.

FIG. 19 is a flowchart illustrating processing of the data processingunit 54 according to the third embodiment. In the following, adescription will be given of the processing of the data processing unit54 with reference to FIG. 19.

In the processing in FIG. 19, first, in step S130, the data processingunit 54 obtains the number of users (C_(max)) of the group X to whichthe user A belongs. Next, in step S132, data processing unit 54 obtainsan unobtained content timeframe i during the content timeframes t_(m1)to t_(m1+n).

Next, in step S134, the data processing unit 54 obtains the degree ofbeing out of ordinary timeframe (q_(i)) calculated using the averagevalue of the PNN transition values of the users who belong to the groupX in the content timeframe i. In this regard, it is possible todetermine whether the content timeframe i is an unordinary PNNtransition section or not by the degree of being out of ordinarytimeframe (q_(i)).

Next, in step S136, the data processing unit 54 calculates the number ofusers (C_(i)) in the group X determined that the content timeframe i isthe unordinary PNN transition section.

Next, in step S138, the data processing unit 54 calculates a ratio(g_(i)) of the other groups having the content timeframe i determined tobe the unordinary PNN transition section. In this case, for example, ifthe content timeframe i is the unordinary PNN transition section in allthe other groups, g_(i) becomes 1.

Next, in step S140, the data processing unit 54 calculates the neutralplus estimation value v_(i) by Expression (3).

v _(i) =q _(i)×((C _(i) /C _(max) +g _(i))/2)  (3)

In this case, if the value v_(i) is a certain value or more, it ispossible to determine that the timeframe is a neutral plus section.

In this manner, in the present embodiment, the neutral plus estimationvalue v_(i) is calculated using, as weights, the ratio (C_(i)/C_(max))of the user determined to be an unordinary PNN transition section in thegroup to which the user to be determined belongs, and the ratio (g_(i))of the group determined to be an unordinary PNN transition section amongthe other groups. Thereby, a higher value is calculated as a neutralplus estimation value in a timeframe in which the PNN transition isdetermined to be unordinary among the users of the same group in common,and the PNN transition is determined to be unordinary among the othergroups in common.

Next, in step S142, the data processing unit 54 determines whether allthe content timeframes have been obtained. If it is determined that allthe content timeframes have not been obtained, the processing proceedsto S132, obtains unobtained content timeframes i, and theabove-described processing is repeated. On the other hand, if it isdetermined that all the content timeframes have been obtained in stepS142, the entire processing in FIG. 19 is terminated.

In the above, as described, by the third embodiment, a neutral plusestimation value v_(i) is calculated in consideration of not only thegroup to which the user belongs, but also the other groups so that it ispossible to estimate a neutral plus section with higher precision.

Fourth Embodiment

Next, a description will be given of a fourth embodiment with referenceto FIG. 20 and FIG. 21. In the fourth embodiment, the data processingunit 54 determines the unordinary PNN transition section using the PNNtransition value by the use of selected data (data of a user having acertain attribute (for example, a person in his fifties who is workingat Shiodome head office, and so)) so as to identify a timeframe having ahigh neutral plus estimation value that is specific to the selecteddata.

The server 10 collects data of a plurality of users, and performsprocessing by selecting data of only users having a user attribute C inorder to identify a timeframe having high possibility of being positive(the timeframe having a high neutral plus estimation value) for theusers with a user attribute (for example, an attribute C) in the timeperiod of the content timeframes t_(m1) to t_(m1+n).

FIG. 20 illustrates processing of the data processing unit 54 accordingto the fourth embodiment. In the following, a description will be givenof the processing of the data processing unit 54 with reference to FIG.20. In this regard, at a stage of starting the processing in FIG. 20, itis assumed that the data processing unit 54 is identifying the timeframe(t_(h1) to t_(h1+j)) indicating a high neutral plus estimation valueusing data of a plurality of users in the same manner as in the firstembodiment, and so on. In this regard, the i-th and the (i+1)-thtimeframes (t_(i) and t_(i+1)) are in chronological order. However,t_(i+1) may be not immediately after t. Also, at t_(i) and at t_(i+1),data may be of cases where separate contents were viewed.

In the processing in FIG. 20, first, in step S150, the data processingunit 54 obtains a timeframe (t_(h1) to t_(h1+j)) indicating a highneutral plus estimation value during the content timeframes t_(m1) tot_(m1+n).

Next, in step S152, the data processing unit 54 obtains the contenttimeframe i during a time period from t_(h1) to t_(h1+j). Next, in stepS154, the data processing unit 54 calculates the average value (S_(i))of the PNN transition values of all the users in the content timeframei. In FIG. 21, S_(i), which was calculated in step S152, is stored inthe field of “original data”.

Next, in step S156, the data processing unit 54 calculates the PNNtransition value (p_(i)) in the content timeframe i of all the users(all the users having an attribute C) who belong to the selected data(assumed to be data X).

Next, in step S158, the data processing unit 54 calculates the averagevalue (P_(i)) of the PNN transition values of all the users who belongto the selected data X in the content timeframe i. In FIG. 21, P_(i),which was calculated in step S158, is stored in the field of “selecteddata”.

Next, in step S160, the data processing unit 54 determines whether thefollowing expression holds or not: |S_(i)−P_(i)|>S_(i)×10%. In thisregard, in step S160, a determination is made of whether there is asufficient difference between the original data and the selected data.Here, if it is determined that the expression is not held, theprocessing proceeds to step S170, whereas if it is determined that theexpression is held (if there is a sufficient difference), the processingproceeds to step S162. In this regard, in FIG. 21, it is assumed to havedetermined that there is a sufficient difference in the timeframet_(h1+1).

When the processing proceeds to step S162, the data processing unit 54determines whether a function of the user's audio/visual action statevariable in the timeframe i is a monotonous decrease function or not.Here, if it is determined that the function of the user's audio/visualaction state variable in the timeframe i is the monotonous decreasefunction, the processing proceeds to step S164, and the data processingunit 54 determines whether S_(i)<P_(i). For example, in the case ofmonotonous decrease in the timeframe t_(h1+1) in FIG. 21, and thus theprocessing proceeds to step S168. When the processing proceeds to stepS168, the data processing unit 54 determines that the timeframe i is atimeframe having a remarkably high neutral plus estimation value in theselected data (refer to the field of “attribute that is remarkablypositive than neutral” in the timeframe t_(h1+1) in FIG. 21). Afterthat, the processing proceeds to step S170. On the other hand, if it isdetermined that S_(i)<P_(i) is held in step S164, the processingproceeds to step S170 without going through step S168.

On the other hand, in step S162, if it is determined that the user'saudio/visual action state variable in the timeframe i is not amonotonous decrease function, the data processing unit 54 proceeds tostep S166. In step S166, the data processing unit 54 determines whetherS_(i)>P_(i). If it is determined that S_(i)>P_(i) is held, theprocessing proceeds to step S168, and in the same manner as above, thedata processing unit 54 determines that the timeframe i is a timeframehaving a remarkably high neutral plus estimation value in the selecteddata, and the processing proceeds to step S170. On the other hand, if itis determined that S_(i)>P_(i) is not held in step S166, the processingproceeds to step S170 without going through step S168.

When the processing proceeds to step S170, the data processing unit 54determines whether all the content timeframes have been obtained. If itis determined that all the content timeframes have not been obtained,the processing proceeds to step S152, obtains an unobtained contenttimeframe and the above-described processing is repeated. On the otherhand, if it is determined that all the content timeframes have beenobtained in step S170, all the processing in FIG. 20 is terminated.

In this regard, in the fourth embodiment, the significant differences ofthe PNN transition values are compared between the selected data and theoriginal data by limiting to timeframes t_(h1) to t_(h1+j) having thehigh neutral plus estimation value. However, the significant differencesmay be compared among the timeframes of all the PNN sections.

In this regard, in the fourth embodiment, the data processing unit 54achieves the functions of a selection unit that selects a part of aplurality of users, and a comparison unit that estimates a timeframehaving a high possibility of the selected user having been in a positiveor a negative state using the detection result of the state of theselected user, and compares with the estimation result of the estimationunit.

As described above, by the fourth embodiment, it is possible to identifya timeframe having a possibility of being remarkably positive on acertain attribute. Thereby, it is possible to identify a part of acontent in which a user having a certain attribute has an interest orattention in particular with high precision.

Fifth Embodiment

In the following, a description will be given of a fifth embodiment withreference to FIG. 22 and FIG. 23. In the fifth embodiment, at the timeof determining the unordinary PNN transition section, the dataprocessing unit 54 determines the unordinary PNN transition section foreach of the timeframes having multiple constant period.

In the fifth embodiment, it is assumed that in the same manner as thesecond embodiment, and so on, a user A of the client 20 is displaying acontent of the content ID=1 on the screen B. Also, in the fifthembodiment, it is assumed that the data processing unit 54 calculates aneutral plus estimation value of the user A during a PNN section k(content ID=1, and the content timeframes t_(m1) to t_(m1+n)). Also, itis assumed that a posture change is used as a user's audio/visual actionstate variable in the PNN estimation value calculation.

Here, posture change (transition of posture) itself does not changesimply from a close side of the screen to a far side in the PNN section.In particular, if the user loses an interest in a content, it is thoughtthat the user often reveals small movements restlessly. That is to say,a little posture movement is sometimes observed at the same time inaddition to transition from a close side to a far side from a screen inthe PNN section.

Accordingly, if a determination is made of the unordinary PNN transitionsection after a PNN transition value is calculated, there are caseswhere a determination result is different depending on which constantperiod is used for comparison. Accordingly, in the fifth embodiment, inthe determination of the unordinary PNN transition section, PNNtransition values are compared using multiple constant period as anobservation timeframe in order to determine the unordinary PNNtransition section.

FIG. 22 illustrates an example of a PNN transition value used in thefifth embodiment. Also, FIG. 23 illustrates an example of comparison ofPNN transition values and determination results of the unordinary PNNtransition section in multiple constant period. In this regard, in FIG.23, a constant period based on the determination of the unordinary PNNtransition section is described together.

In these figures, a comparison is made of how degrees of being out ofordinary timeframes are different in one time constant period and fourtimes constant period by focusing attention on the timeframe t_(m1+k).Here, in order to calculate a neutral plus estimation value, a degree ofbeing out of ordinary timeframe is directly used without a weight in thesame manner as the first embodiment.

In one time constant period, there is not so much difference with anapproximation function value in the case of a neighboring timeframe, andthus in a change of the PNN transition value as illustrated in FIG. 22,it is difficult to understand that a change has occurred in thistimeframe. In contrast, in four times constant period, the PNNtransition value has dropped to a value near to the value at timet_(m1), and thus it is understood that this is a timeframe having arelatively large degree of being out of ordinary.

In this manner, in the fifth embodiment, the degree of being out ofordinary is determined by a constant period in which changes to bedetected occur consecutively so that it becomes possible to detectdesired degree of being out of ordinary. In this regard, there arevarious time constant periods of a change to be detected, and thusdegree of being out of ordinary may be determined in multiple constantperiod, or a comprehensive method, such as a method of comparing degreeof being out of ordinary in all the integer multiples of a contrastperiod, may be employed.

Also, in the fifth embodiment, the PNN transition value of the m-timescontrast period is represented by the average value. However, a localmaximal value or a local minimal value may be used as a representativevalue of the timeframe. Also, a degree of being out of ordinarytimeframe for each of a plurality of one time, two times, . . . andm-times constant period, which was obtained here, is a value calculatedin order to determine the degree of being out of ordinary of one-timeconstant period to be a reference. In the case where it is notpreferable to use the average value of the data of the other timeframesor a local extreme value of the timeframe in order to determine thedegree of being out of ordinary of the timeframe, a degree of being outof ordinary timeframe at m-times constant period may be calculated usinga coefficient that shows the inclination of the line for linearapproximation of a change graph of the PNN transition value in atimeframe adjacent to the m-times constant period, and so on in place ofthe PNN transition value.

As described above, by the fifth embodiment, the degree of being out ofordinary is detected in multiple constant period, and thus even when alarge movement and a small movement are observed at the same time justlike a posture change, it is possible to determine the unordinary PNNtransition section with high precision. Also, in the fifth embodiment, aconstant period to be grounds is also recorded as a determination resultof the unordinary section (FIG. 23), and thus the grounds fordetermining the unordinary PNN transition section (what posture changecauses to determine the unordinary PNN transition section) are apparent.Thereby, when a determination result of the unordinary PNN transitionsection is used, it becomes possible to make use of the determinationresult based on the grounds.

In this regard, in each of the above-described embodiments, adescription has been given of the case where the server 10 and theclient 20 are separate apparatuses. However, the present disclosure isnot limited to this, the server software and the client software may beimplemented in one apparatus, and the one apparatus may achieve thefunctions of the server 10 and the client 20.

Also, the functions of one software system may be distributed in aplurality of apparatuses. For example, a part of the functions of theserver 10 and the client 20, or a part of data may be held separately inthe other apparatus, and so on.

In this regard, it is possible to achieve the above-described processingfunctions by a computer. In that case, the program describing theprocessing contents of the functions of the processing apparatus isprovided. By executing the program on a computer, the above-describedprocessing functions are achieved on the computer. It is possible torecord the program describing the processing contents in a computerreadable recording medium (however, a carrier wave is excluded).

In the case of distributing the program, the program is marketed in theform of a portable recording medium, for example, a digital versatiledisc (DVD) on which the programs are recorded, a compact disc read onlymemory (CD-ROM), and so on. Also, it is possible to store the program ina storage device of a server computer, and to transfer the program fromthe server computer to the other computers through a network.

A computer that executes the program stores, for example, the programrecorded in a portable recording medium or the program transferred froma server computer into its own storage device. And the computer readsthe program from the own storage device, and executes processing inaccordance with the program. In this regard, it is possible for thecomputer to directly read the program from the portable recordingmedium, and execute processing in accordance with the program. Also, itis possible for the computer to execute processing in accordance withthe received program one after another each time the program istransferred from the server computer.

The above-described embodiments are examples of preferable modes forcarrying out the present disclosure. However, the present disclosure isnot limited to this. Various variations are possible without departingfrom the spirit at the scope of the present disclosure.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the inventionand the concepts contributed by the inventor to furthering the art, andare to be construed as being without limitation to such specificallyrecited examples and conditions, nor does the organization of suchexamples in the specification relate to a showing of the superiority andinferiority of the invention. Although the embodiments of the presentinvention have been described in detail, it should be understood thatthe various changes, substitutions, and alterations could be made heretowithout departing from the spirit and scope of the invention.

What is claimed is:
 1. A method of processing information, the methodcomprising: identifying a time span in a period of viewing a contentbased on detection results of the behavioral viewing states of the userviewing the content, the time span being a period, during which abehavioral viewing state of the user is not determined to be a positivestate or a negative state; extracting a time period during which anindex indicating one of the positive state and the negative state of theuser has an unordinary value with respect to values of the other timeperiods in the time span; and estimating a time period, during which theuser has quite possible been in at least one of the positive state andthe negative state, based on the time period extracted by theextracting.
 2. A method of processing information, the methodcomprising: grouping users who tend to perform a similar action among aplurality of users viewing a content, based on detection results ofbehavioral viewing states of the plurality of users viewing the content,the behavioral viewing states including a positive state and a negativestate; identifying a time span in a period of the viewing performed by acertain user among the plurality of users, the time span being a periodduring which a behavioral viewing state of the certain user is notdetermined to be the positive state or the negative state; determining atime period in the time span, during which a behavioral movement that isdetermined to be in a positive state or in a negative state, inassociation with the behavioral viewing state in common with the certainuser and the other users in a first group generated by the groupingincluding the certain user; and first estimating a time period, duringwhich the certain user has quite possibly been in at least one of thepositive state and the negative state, based on the time perioddetermined by the determining.
 3. The method of processing informationaccording to claim 2, wherein the first estimating estimates a timeperiod, during which the certain user has quite possibly been in atleast one of the positive state and the negative state, based ondetermination of whether the time period determined by the determininghas a common tendency in a second group different from the first groupgenerated by the grouping.
 4. The method of processing informationaccording to claim 2, further comprising: selecting part of the usersamong the plurality of users based on information other than thedetection results; second estimating a time period, during which thepart of the users have quite possibly been in at least one of thepositive state and the negative state based on detection results on thepart of the users among the detection results; and comparing a result ofthe first estimating and a result of the second estimating.
 5. Aninformation processing apparatus comprising: a memory; and a processorcoupled to the memory and configured to: execute grouping of users whotend to perform a similar action among a plurality of users viewing acontent, based on detection results of behavioral viewing states of theplurality of users viewing the content, the behavioral viewing statesincluding a positive state and a negative state, identify a time span ina period of the viewing performed by a certain user among the pluralityof users, the time span being a period during which a behavioral viewingstate of the certain user is not determined to be the positive state orthe negative state, determine a time period in the time span, duringwhich a behavioral movement that is determined to be in a positive stateor in a negative state, in association with the behavioral viewing statein common with the certain user and the other users in a first groupgenerated by the grouping including the certain user, and execute firstestimating of a time period, during which the certain user has quitepossibly been in at least one of the positive state and the negativestate, based on the time period determined.
 6. The informationprocessing apparatus according to claim 5, wherein the processor isconfigured to execute the first estimating of the time period, duringwhich the certain user has quite possibly been in at least one of thepositive state and the negative state, based on determination of whetherthe time period determined by the determining has a common tendency in asecond group different from the first group generated by the grouping.7. The information processing apparatus according to claim 5, whereinthe processor is configured to: select part of the users among theplurality of users based on information other than the detectionresults, execute second estimating of a time period, during which thepart of the users have quite possibly been in at least one of thepositive state and the negative state based on detection results on thepart of the users among the detection results, and compare a result ofthe first estimating and a result of the second estimating.